Azure Image OCR #228
Merged
Azure Image OCR #228
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 8 commits
October 1, 2020 13:04
…erate attatchment metadata hook, that will kick off OCR processing. Add a new REST endpoint that will also utilize this same function. Add an OCR class that does all the processing. Currently only process PNG images.
…odal and single media edit screens. Bypass mime check if recan button or option is used. Add filter around approved media types
…ensions. Add support for the four file types azure supports. Some code cleanup
…out supported image types besides PNG for now
… button, to populate the image description after success.
…id errors. Remove local only code
…ng. Pass the previously run image scan into our OCR function and utilize that to determine if the image needs OCR run or not.
|
Noting here that @dinhtungdu is going to work on scaffolding in the Gutenberg bits |
…f should only turn off the automatic scanning, not the manual scanning
This was referenced Oct 21, 2020
Closed
…encodings properly, so we don't end up with weird characters. Keep track of if we need to modify the content or not, so if we don't, we can just return the original and not risk messing anything up. Minor code formatting
…at WordPress itself does in other places. This will hopefully be more lenient across environments and encoding types
helen
reviewed
Oct 24, 2020
|
This is feeling pretty good to me for a first run. I think what we might want to do is add a field to |
helen
reviewed
Oct 24, 2020
…nto feature/azure-image-ocr
|
@helen I fixed the REST API response issue. This PR is working on my live site. |
…art cropping button. Make sure we have HTML elements before trying to add event handlers to them
This way any additional line breaks will keep the single block with its ID intact. Also massages the message in the modal.
Didn't realize verse was a pre without line wrapping :(
CSS isn't yet building into `dist`
This is in-progress, see code comments for needs
… the classname we want to add. Minor linting fixes
This allows for a number of customizations, such as being able to store the text results in a different field in case you use post_content extensively in the editorial workflow already, or set other meta, or update the alt text based on the text results, and so forth. Also adds the full $scan data as context for the `classfai_ocr_text` fitler.
…nto feature/azure-image-ocr
… only used for editor highlighting
fix: switch to use internal style
helen
approved these changes
Nov 2, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the Change
Add support for Azure image OCR.
Adds a new setting that is turned off by default but can be turned on to run OCR scanning on images. When an image is initially uploaded, we run a few checks on that image. First, we check if the image matches one of the supported types (JPEG, PNG, GIF, BMP). If so, we then check if we have a previous image scan and if so, we check if we have either the handwriting or text tags set, with a high confidence level (above 90%). If all those checks pass, we then run OCR scanning.
If we get a successful response back, we then parse the text out of that response, save that text to
wp_contentfor that image and then save the full response to post meta.We also add scan/recan functionality to the media modal and single media edit screens. If a scan is run from here, we bypass our checks, as we assume if someone is manually starting a scan, they don't care about the checks.
Also added some better error handling to the scan/rescan buttons, so if a scan fails, we don't continue to show the loading icon but instead remove that, keep the button disabled and change the button text to say error (ideally we would add even better error handling in a separate PR).This is now in a separate PR: #231TODO: last piece is to add
describedbytext into the content when needed. Need to support the block editor and the classic editorAlternate Designs
Benefits
Images with text can now have that text automatically read and then inserted into the content, with the proper
describedbytags. This helps provide more context for images, especially images that are screenshots of text (like social media posts).Possible Drawbacks
With this feature turned on, will cause a slight slowdown (seems like roughly 5 extra seconds), in image processing.
Verification Process
Checklist:
Applicable Issues
#111